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Abstract; To interface between the one-dimensional scanning of an image, and the application of a two- 
dimensional operator, an intermediate storage is required. For a square image of size n 2 , and a square 
operator of size m 2 , the minimum intermediate storage is shown to be n • (m — 1). An interface of this size can 
be conveniently realized by using a serpentine delay line. New kinds of imagers would be required to reduce 
die size of the intermediate storage below n • (m — 1). 
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1. Introduction 

Image processing often requires the application of a two-dimensional operator over the image. Imaging 
devices usually scan the image along a one-dimensional path, and produce a stream of data points. In such 
cases, an intermediate storage is required, to interface between the one-dimensional scanning and the applica¬ 


tion of the two-dimensional operator. 

In this paper, the interface problem will be examined. In particular, we shall determine the minimum 
size required of the intermediate storage. For a square image of size n 2 , and a square operator of size m 2 , the 
minimum intermediate storage is shown to be n • (m — 1). For small-size operators the intermediate storage 
may not add substantially to the overall size of the system. For example, a recent two-dimensional CCD 
scanner developed by Texas Instalments (Hall & Awtrey, 1979) incorporates the intermediate storage required 
for a 3*3 operator on the scanner chip itself. For larger operators, however, the size of the intermediate 
storage becomes significant For example, a recent CCD convolution chip constructed by Hughes is designed 


to support a 26 * 26 operator (Nudd el al , 1979). It is intended to be applied to images of 1000 points on a 
side, in which case an intermediate storage of 25000 image points would be required. In such cases the size of 
1% system is determined almost entirely by the intermediate storage requirement. Given die theoretical tower 
bound on the size of this storage, it is suggested that new kinds of imagers would be required for systems for 
which size is a dominant consideration. 


2. Using a serpentine delay line of size n • (m — 1) 

A simple method for interfacing a onc-dimcnsional scanner to a two-dimensional operator is shown in 
Figure 1. The camera c scans the image (assumed to be a square of n 2 points) row by row. The intensity values 
measured by the camera are fed into a serpentine delay line (S in Figure 1), consisting of rn — 1 rows, each 
of length n. The serpentine delay line is tapped in m — 1 positions that arc connected to the two -dimensional 
operator M (of size m 2 ). In addition, there is a direct line from the camera to M, so that there arc a total of m 
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input lines to M. 

After die first m — 1 rows have been scanned, they fill the intermediate storage S, and the first column 
of these rows resides in the rightmost column of S, ready to be delivered to M. Tire scanner then reads point 
(m,l) in the image, and the first column of m points (point (1,1) to (m,l)) is transferred to M. Point (m,l) 
also enters the delay line, shifting each point in S by one position. The point in the lower-right corner of S 
is dropped from the memory and lost. Whenever a new point is now read by the scanner, a column of m 
points is transferred to M. After having scanned m points from the m th row, the entire upper-left square of m 2 
points resides in M, and the operator is applied to it. When the scanning of the entire image is completed, the 
operator has been applied to all the squares of size m 2 in the image. 

The intermediate storage S in this scheme is required to hold n • (m — 1) image points. The next section 
will establish that this requirement cannot be reduced. Even if one uses arbitrary scanning pattern (e.g., the 
image may be divided into sub-blocks, and each block scanned and operated upon separately), and arbitrary 
interconnections between S and M (i.e., M may have access to all the points in S ), the minimal intermediate 
storage would still be n ■ (m— 1). 
















3. The minimal intermediate storage is n • (m — 1) 

Let the size of the intermediate storage be S. In this part we wish to show that S > n ■ (m — 1). 

Notation: We shall think of the image as a grid of n * n squares, n > 2. Each square in the grid is also 
called a point in the image. A two-dimensional operator of size m* m is a function of m 2 arguments. The 
function is to be applied to every square of size m 2 points in the image. We shall assume that the size of the 


operator (m) is at most f. Finally, for a set P of points, ||P|| will denote the number of points in P. 

Definitions: Two distinct points in the image are called partners if they can be covered simultaneously by 
a square of size m 2 . That is, their distance in the image in either the horizontal or the vertical direction is 
at most m points. Similarly, the partners of a set P of points are die partners of all the points in P, but not 
including the points in the set P itself. That is, 


partners(P) — M partners(i) — P 

it P 



When the scanner has read k points, k > S, some of the k points are no longer in tire storage, and can no 
longer be retrieved. Let P be the set of points that have been scanned, but are no longer in the storage. It is not 
difficult to see that at this stage the partners of the set P must reside in die intermediate storage. The reason is 
that if points p and q are partners, then there must exist a stage at which both p and q are in the intermediate 
storage. We can now obtain a lower bound on the size of the intermediate storage by estimating die number 
of partners of a set of k points. We shall establish the lower bound on the size of the intermediate storage by 
showing that for a certain choice of k, the number of partners is at least n ■ (m — 1). 

Proposition I: Let k be the smallest integer such that k > For this choice of A;, any set of A; points has 
at least n(m — 1) partners. 

Proof: By induction on m, the size of the two-dimensional operator. For m — 1 the claim holds, since 
die number of partners is zero. To prove the induction step we shall show that, in an image of size n 2 , when 
the size of the operator is increased from m = M to m = M + 1, (assuming M -f- 1 < §), the set of partners 
is increased by at least n new points. To prove the induction step, wc shall use the following two definitions. 







Definition I: Let I be the image, and P a collection of points in the image. A neighbor of the setT is a 

point in 1 - P that shares an edge or a vertex on the grid with a point in P. 

Definition2: In a grid of size n 2 , a set of A: points is called admissible (and the grid itself is admissible) if: 
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n 2 < k < -n 2 for n > 2 
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< k < -n 2 for n = 2 


Comment: Let P be the set of the k original points, and all their partners for ail operator of size M. When 
the operator increases in size to M 1, all the neighbors of P become new partncis of the k original points. 
To prove proposition 1 it will be sufficient, therefore, to show that the set P has at least n neighbors. This will 

be tire goal of proposition 2. 

Without loss of generality, we can assume that the set P is admissible. Since P contains the k original 
points, clearly 4 < ||P||. If ||P|| > ifn 2 , then the k original points have at least f « 2 —k partners. 


points, clearly < ||P||. U \\F\\ > -then me K original points nave at lease -.poiutcra, 

P' t since k < *£ 4-1, and f > M -f 1, it would follow that tile /c original points have at least n • m 
partners. Proposition 1 would therefore hold for the operator of size M -f-1, and tire induction step would be 
unnecessary. 

Proposition 2.' In a grid of jt 2 points, any admissible set of K points has at least n neighbors. 


- Proof: By 


on n, the size of the grid. For n — 2 the proposition holds, since K — 1 or K — 2, 


and in either case there are at least 2 neighbors. We next assume that proposition 2 holds for n =- N, and 




+ 1 . 


The general structure of the proof will be as follows. For convenience of reference, the K points will be 


called tire black points, and the remaining points white. In these terms, we wish to show that in a grid of size 


{N + i) 2 any set of K points, 


l) 2 < K < : \{N + l) 2 , has at least N 4-1 white neighbors. To use 


the induction hypothesis wc^ shall provethe following claim: 





Proposition 3: It is possible to remove a row and a column from the grid in such a way, that the resulting 
grid of size N 2 will be admissible. 

From the induction hypothesis, in the reduced grid the set of black points has at least N neighbors. The 
proof will be concluded by showing that when the missing row and column arc re-introduced, the number of 
neighbors of the black set is increased by at least one. 

Proposition 3.1: From an admissible grid of size (N -f l) 2 it is possible to remove a row and a column, so 
that in the remaining grid of size iV 2 the number of black points will be less than \ N 2 . 

Proof: Let a N+l denote the fraction of black points in the original grid. That is, the number A' of 
black points is equal to a/v+i • {N + l) 2 . Similarly, q/v is the fraction of black points in the reduced grid. In 
these terms, we assume in proposition 3.1 that \ < a/v+i < §, and wish to establish that a/v < f. 

Let Bij denote the total number of black points in row i and column j. Bij is the (number of black points 
in row i) + (the number of black points in row j). The difference between B t] and B^ is that if point (ij) is 
black, it is counted twice in Bij but only once in Bij. Let B — Bij, B = Bij. 

There are cin+i{N -f- l) 2 black points in the grid, and each one is counted 2 (N -f- 1) times in B, 
therefore: 

B — 2(N 1) • a/vq_i((V + l) 2 (3) 

In B each black point is counted one additional time, compared with B, therefore: 

B = B-a N+l (N + 1) 2 (4) 

There are (N -f l) 2 terms B^ contributing to the sum B, therefore the average contribution is: 

B 

(7V+T)2 = 2 q /v+i(N + 1) — Q/v+1 (5) 

There must exist a term B l3 that contributes at least the average contribution. Proposition 3,1 will therefore be 
satisfied if: 

Q/v + i(N + l) 2 -(2Q /v+ ,(N-f l)-a N+1 )< ^N 2 (6) 
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If this inequality does not hold, then: 


a : v ( .|(N -f I) 2 — (2a ; v-i_i(jV + 1) — 





But tliis implies a/v-j-i > which is a contradiction, since \ < Qn+i < 1- 8 


Proposition 3.2: In the conditions of proposition 3.1 it is possible to remove a row and a column so that in 
die reduced grid q/v > 

The proof will be omitted, since it is entirely analogous to the proof of proposition 3.1. 

Comment: The row and column chosen in propositions 3.1 and 3.2 are not necessarily the same. If, in 


proposition 3.1, gjv ] • or if, in proposition 3.2, o\ j|, then proposition 3 is established. Proposition 
3 may still fail, however, to be satisfied if in proposition 3.1 o/v < \ and in proposition 3.2 a,v 2> §• L is 


required therefore to show that in tliis case we can still remove a row and a column such that in the reduced 
grid the set of black points is admissible. 


Let the row and column chosen in proposition 3.1 be row i and column j. The row and column chosen 
in proposition 3.2 will be denoted row k and column /. Let us now remove from the grid row i and column /. 
Since the definition of an admissible set is slightly different for N = 2, we shall start with the case N > 3. In 
this case, proposition 3 would fail to be satisfied if either: 

o n+1 (N + 1) 2 -^<|^ 2 P 


or: 

a N+l {N-\-l) 2 -~B i{ > ^N 2 (9) 

Let us examine the first case. We may assume that 


q n+ i(N -f l) 2 “ Bki > ^N 2 









(see the comment above). From (8) and (10) we conclude that: 

( 11 ) 

Bu — B k .i is the difference in the number of black points between rows i and k, after removing the I th column, 
hence Bu — B k i < N. '1'his, together with (11), implies N < 2, in contradiction with the assumption that 
iV > 3. We conclude that for N > 3 (8) is impossible, and the inequality ayv+i(N + l) 2 — Bn > \N 2 
must hold. To include die case N — 2, a similar argument establishes that q^ + 1 (N -f l) 2 — B it > \N 2 
must hold for N > 2. 

The case in (9) can be handled analogously, yielding 

Bij - Bu > ^N 2 (12) 

which again cannot hold for N > 2. 

In conclusion, die fraction a N satisfies | < a N < | for N > 3, and ^ < a N < \ for N — 2. r rhe 
set of black points in the reduced grid is therefore admissible. | 

Proposition 3 has established that a row and a column can be removed from die original grid of size 
{N -f l) 2 , resulting in an admissible grid of size N. From die induction hypothesis, in the reduced grid the set 
of black points has at least N neighbors. To show diat the set of neighbors in the original grid is larger (by at 
least one), we shall use the following claim: 

Proposition 4: proposition 3 can be satisfied by removing a row and a column that are not entirely black. 

Proof: It is sufficient to show that proposition 3.1 can be satisfied with a row and a column that contain 
at least one white point. We can assume without loss of generality that the original grid contains die maximal 
possible number of black points, since if proposition 3.1 can be satisfied (with a row and a column that are 
not entirely black) for a maximal number of black points, it can also be satisfied for grids with fewer black 
points. For N -f 1 = 2m (an even number), the maximal number of black points is |(N -f l) 2 — 1. For 
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/V 1 — 2m -f 1, the maximal number is |(N -j- l ) 2 — §. We shall examine the latter case (the fiistcan be 
established analogously). 

let r be the number of completely black rows in the original grid, c the number of completely black 
columns. We can assume that r > 1, c > 1, since otherwise proposition 4 followes directly from proposition 


thatc< 


a N +i < f, either c < or r < ^ 


must 


• prOpUMUUIl H- lUUUWVb uvm 

hold. Without loss of generality we $ 


shall assume 


The r black rows and c black columns contribute a total of 


(r + c)(N -f 1) — re 


black points. The number of black points in the remaining rows and columns is therefore: 


N + l ) 2 — 7 — (r -f c)(N 4 - 1) 4 ~ re 
4 4 


(14) 


This number of black points is divided among JV + l- e columns, hence the average per column (denoted 


is: 


A = 


|(N 4-1 ) 2 | — ( r "l" C ){N 4-1) + rG 

' . 1 N + l-e ' 


must exist a column which contains at least r 4~ A black points. By removing this, column and one 


black rows we remove at least: 


(N+l) + r+4-l 


black points. It therefore remains,to establish that: 


|(N 4-1 ) 2 ~ l ~ (N + r 4- A) < 


( 17 ) 


Inequality (17) reduces to: 


e 


Fore < —■■■ inequality (18) holds, therefore proposition A is established. 




Corollary: using the same argument, proposition 3.2 can be established with a row and a column that 
are not entirely white. Combining the two claims, it is straightforward to establish that proposition 3 can be 
satisfied by a row and a column that arc neither entirely black nor entirely white. 


Proposition 5: The set of black points in the original grid has at least N -f 1 neighbors. 

Proof: From propositions 3 and 4, we can remove a row R and a column C (neither entirely black nor 
entirely white) from the original grid, and obtain an admissible grid. From the induction hypothesis, the set of 
black points in the reduced grid has at least N neighbors. We have to show that by re-introducing R and C 
the number of neighbors increases by at least one. 


Since R and C contain both black and white points, they contain at least one "internal neighbor", i.e., a 
white point that borders a black point, where both points are in R (J C ,We shall show that without the inter¬ 
nal neighbor the number of neighbors in the original grid is at least N, hence the total number of neighbors is 
at least N -f- 1, as required. 


If a given point p is a neighbor of the black set in the reduced grid, it may or may not remain a neighbor 
when the missing row and column are re-introduced. Let us examine first the case where p is not a neighbor 
of the black set in the original grid. In this case p must be surrounded in the original grid by white points (see 
Figure 2). In this configuration, point q in R becomes a new neighbor, since at least one of the points a, b,c 
must be black. Note that q cannot be an internal neighbor. (Comment: in Figure 2, point p borders the re¬ 
introduced column R. The argument is unchanged when it borders C instead of-R. The case where p borders 
both R and C requires a slight extension of the argument, which will not be elaborated.) 

We conclude that there exists a one-to-one mapping from the neighbors in the reduced grid into the 
neighbors in the original grid, and that this mapping docs not include the internal neighbor. The number 
of neighbors in die grid of size N -(- 1 is therefore at least N -J- 1. ITus establishes the induction step in 
proposition 2, and concludes the proof of propositions 2 and 1. | 









4 * 







4. Reducing the size of the intermediate storage 

The lower bound established in the preceding section implies that the intermediate storage may play a 
dominant role in determining the size of the system. The intermediate storage can, however, be reduced by 
scanning the image more than once. Suppose, for example, that s rows are scanned in parallel. The first scan- 
path scans rows 1 to s, the second scans rows 2 to s + 1, etc. Each image point is scanned in this scheme s 
times. Using the serpentine delay line of Section 2, it is possible now to connect s lines directly to the two- 
dimensional operator M. The intermediate storage S will then have only (m — s) rows. This scheme gives 
a simple (though not optimal, in terms of the storage size) method of reducing the size of the intermediate 
storage ton - (m — s), where s is the number of times each point is scanned. In particular, by having s = m 
the intermediate storage becomes unnecessary. 
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